Orbital Nematic Instability in Two-Orbital Hubbard Model: A Renormalization-Group Study 
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Motivated by the nematic electronic fluid phase in S13RU2O7, we analyze the (dxz, dj^-orbital Hubbard 
model by the one-loop renormalization-group method. It is confirmed that the present model exhibits the ferro- 
orbital instability near the magnetic or superconducting quantum criticality, due to the Aslamazov-Larkin type 
vertex corrections. This mechanism of orbital nematic order presents a natural explanation for the nematic order 
in SraRu207, and is expected to be realized in various multiorbital systems, such as Fe-based superconductors. 
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A bilayer ruthenate compound Si"3Ru20y has attracted 
much attention since it exhibits a unique quantum critical be- 
havior JjJ-Q]. Field-induced antiferromagnetic quantum crit- 
icality is observed by the many experiments like the NMR 
measurements jg]. Surprisingly, the magnetic quantum crit- 
ical point is hidden by the formation of the nematic elec- 
tronic liquid phase at very low temperatures (~ IK). The 
orientational-symmetry breaking in the nematic phase is con- 
firmed by the large anisotropy of in-plane resistivity 0.0,0]. 
Thus, the quantum criticality and nematic phase formation are 
intimately linked in this material Q]. 

The band structure of the ruthenate oxides are composed of 
the Ru t2 g (d xy , d xz , and d yz ) orbitals. For the microscopic 
understanding of the nematic phase in Sr3Ru207, a large num- 
ber of theoretical works have been devoted. The spontaneous 
violation of the C4 symmetry of the Fermi surface (FS), i.e., 
Pomeranchuk instability, had been frequently discussed by fo- 
cusing on the van Hove singularity. The scenario of the single- 
band Pomeranchuk instability was originally proposed by us- 
ing the renormalization-group (RG) method |8(|, and have 
been analyzed by the mean-field an d perturbation lfl3ll 

studies. However, the temperature-flow (T-flow) RG scheme 
lfl4l[l5tl indicates that the nematic instability might be hidden 
by magnetic instabilities. Thus, the possibility of the single- 
band Pomeranchuk instability is not settled yet. 

Alternative theoretical route to elucidate the nematic phase 
is to focus on the two d xz and d yz orbitals, which give rise 
to quasi-one-dimensional (Q1D) a and j3 bands. It has been 
pointed out that the nematic state is described as an orbital 
ordering ((n xz ) ^ (n yz ))- This ferro-orbital-order scenario 
has been analyzed within the mean-field-level approximation 
by focusing only on the q = modes lfl6l - [l8ll . However, the 
random phase approximation (RPA) leads to the occurrence 
of the antiferro-orbital order because of the nesting of the FS 
IU9I1 . Therefore, the theoretical analysis beyond the RPA is 
required. 

Recently, similar nematic phase in Fe-based superconduc- 
tors, which is also a multiorbital system, attracts great atten- 
tions 1 2(], 21 ] .Up to now, both the spin nematic [22] and or- 
bital nematic 12311 theories had been proposed. The latter the- 
ory pointed out the importance of the vertex correction (VC) 
in the nematic order. However, they studied a limited num- 



ber of VCs, so the importance of VCs should be clarified by 
other unbiased theoretical techniques. For this purpose, the 
RG treatment is quite suitable because the RG method enable 
us to perform the systematic calculations of higher-order VCs. 

In this paper, we propose a novel mechanism of the nematic 
electronic state related to the orbital degrees of freedom, by 
applying the one-loop RG method to the (d xz , d yz )-orbital 
Hubbard model for the first time. We find that the strong or- 
bital nematic fluctuation, i.e., orbital Pomeranchuk instability, 
emerges near the magnetic or superconducting quantum crit- 
icality due to the VCs. The present study indicates that the 
two-orbital single-layer Hubbard model is a minimal model 
to describe the orbital nematic order realized in Sr3Ru207. 

We consider a single-layer {d xz , dj /z )-orbital model, where 
the orbital index /1 = 1 and 2 refer to d xz and d yz , respec- 
tively. The tight-binding Hamiltonian with tetragonal symme- 
try is given in the form 

H o = E E tf^L^v. (d 

k,(T — 1,2 

where = — 2tcosk x — ft, = — 2fcosfc„— ft, and^ 2 = 
^fe 1 = sin k x sin k y , with p, being the chemical potential. 
We put (t, t') = (1, 0.1) which corresponds to Ru-oxides. We 
describe the electron-electron interaction by 

flint ' E " • /,.-'' ../,.. • ''^ // ,..; , 

+ J E C jA,<r C j,2,a c j,2,cr' C j,l,(T' 

+ J E(4i,tS\2,T c li,iS\24 + H - c 0, (2) 
J 

where U and U' are intra-orbital, inter-orbital Coulomb in- 
teractions, respectively, and J represents the exchange and 
pair-hooping interactions. Throughout the paper, the condi- 
tion U = U' + 2 J is assumed. 

The energy band structure and the FSs obtained from H 
[Eq. (Q])] are shown in Fig. Q] The a band forms a hole-like 
FS centered at (tt, tt) while the /3 band forms an electron-like 
FS centered at (0,0). Both bands have flat and nested FSs 
and thus the Peierls fluctuations due to the presence of one- 
dimensional FSs can be expected. 
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FIG. 1: (Color online) (a) Band structure of Ho for n = 1.2. The 
linearized band dispersions are shown by the dashed lines. The low- 
energy excitations of electrons (l^fc 111 ^ 31 ! < Ao) are denoted by the 
shaded area, (b) The patch index (1 ~ 8) on the FSs. (c, d) The dia- 
grammatic representation of the scaling equations for the three-point 
vertex R and the susceptibility x> where I is the scaling parameter. 
The slashed line represents an electron propagation having the en- 
ergy on the shell A t+M < < A;. 



We apply the RG method to this two-band system. The 
scattering processes of electrons having energies less than the 
bandwidth cutoff A are integrated within the one-loop RG 
scheme 112411 . The low-energy parts of band structure are de- 
noted by the shaded area in Fig. Q] (a). In order to analyze 
low-temperature properties accurately, we linearize the band 
dispersion within the cutoff scale Ao and change the k sum- 

J L 

mation into the energy (£) integration H25H . In the bandwidth- 
cutoff RG scheme, the £ integration is replaced by the energy- 
shell integration Ai + di < |£| < A;, where A; = Aoe~' and 
I is the logarithmic scaling parameter. We can analyze low- 
temperature (T) behavior by integrating the RG equations up 
to the scale I ~ ln(Ao/T), unless the one-loop RG breaks 
down due to the divergence of coupling constants. The lin- 
earized dispersions, which are shown by the dotted lines in 



Fy(^) [by ~ bp „((/>)] where v = a, /3 represents the band 



Fig. [T](a), are given by £}! noai 

index. Here, we have introduced the azimuthal angle <fi (see 
Fig.HJ and the "radius" of the FS kp u (4>) lE5ll . The neglected 
higher energy contributions (> Ao) would be less important 
for qualitative results, although this simplification might lead 
to slight underestimation of orbital and spin susceptibilities. 

In the next step, we discretize the FSs into patches and in- 
troduce the patch-index dependence of vertex functions. Since 
the present system has flat FSs, we expect that k dependence 
of vertex functions along the flat FSs would be small. In the 
present analysis we introduce four patches per each band: the 
total number of patches is N p = 8. The patch label is shown 
in Fig. [T](b). The FSs on the even (odd) number of patches 
are composed of the d xz (d yz ) orbital completely. In conven- 
tion, we ignored the Matsubara-frequency dependence of the 
vertex function, and the Matsubara frequencies of the external 
field operators are set to iuj rn —> 0. The external momenta of 



vertices are also projected onto the points shown in Fig.[TJ(b). 

In order to analyze the dominant fluctuations in the present 
two-orbital system, we calculate the susceptibilities by us- 
ing the RG method. Since this system has flat FSs, the spin 
susceptibility X s (q) is expected to be enhanced at the nest- 
ing vector q — Q. The main purpose of the present pa- 
per is to analyze the quadrupole (orbital) susceptibility at low 
temperatures. In the present model, there are two irreducible 
quadrupole operators ESQ: 



3 x 2_ y2 = X ;<•;.,.„'%;.., - ct 2)CT c i)2l a) = n jtl - n Jl2 ,(3) 

tr 

3 xy = X/'''. I + C I2, CT c jM,(t), (4) 

(7 

where j is the site index. The quadrupole susceptibility per 



spin is given by (7 



y and xy) 



l) = lf dT ( T rOy(q,T)d 7 (-q,0)) 1 (5) 



where r is the imaginary time and j3 = l/(kgT). Hereafter 
we consider only the case 7 = x 2 — y 2 and denote x q (q) = 
X^-v (9)' since Xxy(Q) is ver Y small. The divergence of 
X q (q — 0) reflects the emergence of the orbital nematic state 
{{n X z) 7^ (%«})> which is consistent with the nematic phase 
in Sr3Ru20y. In addition, we analyze the charge susceptibility 
X°(<?) = I Jo ' d T (T T n(q,T)n(—q,0)) where n(q,r) is the 
charge density operator. 

The RG equations for the susceptibilities are shown in Figs. 
[T](c) and (d), where R and T are the three- and four-point ver- 
tices, respectively 11241 12511 . In the one-loop RG method, T is 
calculated by solving parquet equations [24]. For the initial 
values of the susceptibilities, we have used the constrained 
RPA (cRPA) results, X z \i=o = Xcrpa (« = s ' ^ c ) Si- The 
cRPA is obtained by the "constraint" non-interacting suscep- 
tibility x° c , which is given by omitting the low-energy inte- 
gration less than Ao. The temperature dependence of x z is 
mainly caused by the four-point vertex T since cRPA shows 
weak temperature dependence. In the present numerical study, 
we put the electron filling n = 0.8, where the best nesting vec- 
tor is given by Q ps (OAtt, 0.47r). We choose the bandwidth 
cutoff A = 0.25. 

In the RPA without VCs JHH, Xrpa(<?) is mainly en- 
hanced by U (W), and both of them have peak structures at 
q = Q due to the nesting of FSs. Xrpa(Q) i s a l wa y s larger 
than Xrpa(Q) f° r me realistic parameter U > U' . We stress 
that XrpaC*? = 0) exhibits no singularity, meaning that the 
nematic order is not realized in the RPA. In the following, we 
will show that x q (<Z = 0) given by RG is strongly enhanced 
because of the VCs. 

First, we focus on the case with U = U'. The T depen- 
dencies of the spin, quadrupole, and charge susceptibilities 
are shown in Fig. |2](a). For comparison, the non-interacting 
results are also shown. Due to the interaction effects, both 
X s (Q) an d X q (0) are enhanced, while x°(0) is suppressed. 
The most striking feature is the critical enhancement of x q (0) 
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FIG. 2: (Color online) (a) Temperature dependencies of x s (Q)' 
X q (0) and \ c (0) for n = 0.8. The solid lines represent the RG 
results, and the non-interacting results (dotted line) are also shown, 
(b) The susceptibilities obtained by the RG method in the logarithmic 
scale of T, for several choices of U and U' The dashed curves are 
the guides for eyes. 



at low temperatures, which cannot be obtained from RPA. We 
stress that the enhancement in x q (<7) is restricted to the q re 
region, indicating the emergence of the orbital nematic order. 

The T dependencies of the susceptibilities given by the RG 
method are shown in Fig. [2] (b), for several choices of U . 
The spin susceptibility X s (Q) is enhanced by the combined 
effect of the FS nesting and the on-site interaction U, from 
high temperatures. On the other hand, X s (Q) is almost T 
independent below T* ~ 10~ 2 due to the effect of nesting 
deviation. Such a behavior is reminiscent of the Peierls in- 
stability in Q1D systems [28]. The quadrupole susceptibility 
X q (0) starts to increase at T ~ 10~ 2 and shows a divergent 



behavior at T re T qua d. In the one-loop RG method, precise 
calculations of the q = mode in susceptibilities tend to be 
difficult at low temperatures, in case that the coupling T; di- 
verges before the convergence of the I integration in x z (0) is 
achieved 

MM. 

For this reason, we show the susceptibili- 
ties for T > T qua d, where the one-loop RG method is quali- 
tatively reliable. In contrast, x c (0) shows no anomaly even at 
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FIG. 3: (Color online) The U'/U dependencies of X q (0) for several 
choice of temperatures. X s (Q) an d X c (0) are a l so shown for T = 
10" 4 , which are almost temperature independent for T < T* . The 
dashed curves are the guides for eyes. 



T ~ Tq Ua d, which would ensure the reliability of the singular 
behavior in x q (0). For larger U, the critical enhancement of 
X q (0) shits to high temperatures. For U — 1.8, T qua d is in- 
creased to 10~ 5 ~ 10~ 4 , which corresponds to ~ IK if t = 1 
eV. 

We stress that the singular behavior in x q (0) is also realized 
for U'/U = 0.9, as shown in Fig.|2](b). Thus, high T qua d can 
be obtained even for J > in the strongly-correlated cases. 
The U'/U dependencies of x q (0) f° r U — 1.5 and 2.0 are 
shown in Fig. [3] with several choices of T. At T > 10 -2 , 
all susceptibilities exhibit linear dependencies in U' , since the 
VCs are not important at higher temperatures. At low temper- 
atures, in contrast, the strong enhancement of x q (0) is seen 
for U'/U > 0.8, which corresponds to J/U < 0.1. 

Here, we discuss the reason why x q (?) is critically en- 
hanced only for q s» in the RG analysis. The enhancement 
of the q « mode is a strong hallmark of the dominant contri- 
butions of the Aslamazov-Larkin (AL) type VCs, since these 
VCs are known as the enhancement mechanisms of q « 
susceptibilities. As shown in Refs. 0and|27l the AL term 
due to the magnetic fluctuations [Fig.|4](a)] gives the enhance- 
ment of x q (0). In addition, we will show the importance of 
the AL term due to the superconducting fluctuations [Fig. [4] 
(b)], which is not discussed in Refs. |23| andl27l For this pur- 
pose, we introduce the effective interaction for ferro-orbital 
fluctuations r quad and that for superconducting fluctuations 
r scrf , which give the quadrupole and d-wave superconduct- 
ing susceptibilities: 



rqUad = ^E[ 2r ^P'^'P')-r(p,p';p',p)]O p O p ({5) 

p p,p' 



where O p is the form factor for 7 



(7) 



y 2 quadrupole: 



-1(— 1) for odd (even) number of the patch index p. 
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FIG. 4: (Color online) (a) The magnetic AL term and (b) the super- 
conducting AL term, where x^C 8 ) is superconducting propagator. 
They are proportional to J^ fe x z (k) X z {k + Q) ( z ~ s i SC X which 
takes maximum value at q = 0. (c) The scaling flows of the effec- 
tive interactions T quad and r SCd at T = 1CT 10 . The large negative 
value of pi uad ^ sc<i gives the enhancement of the corresponding sus- 
ceptibility, (d) T dependence of X q (0) and X s (Q) for U = U' > 2. 
Here, we show results only in the temperature region where the one- 
loop RG method is qualitatively reliable. 



Also, V p is the d-wave form factor: T> p = +1(— 1) forp = 
1, 3, 6, 8(2, 4, 5, 7). The patch indices p and p are related by 
the inversion symmetry. The initial values are r quad |; = o = 
(3U - W)/2 and r scd | i=0 = 0. 

The scaling flows of r quad and r scd are shown in Fig. [4] 
(c) lf30h - The obtained r quad weakly depends on I at high en- 
ergies, while it exhibits a steep increase in magnitude when 
A; ~ T qua d. This behavior gives the divergent behavior in 
X q (0) observed in Fig. [2] Moreover, T° Cd also shows a steep 
increase in magnitude at A; « T qua d: This fact means that 
the development of x q (0) is closely related to the supercon- 
ducting fluctuations. From the diagrammatic arguments, we 
conclude that the major contribution is given by the supercon- 
ducting AL term in Fig.|4](b), which represents the coupling 
between the quadruple and superconducting fluctuations. This 
superconducting-fluctuation-driven orbital nematic order had 
not been pointed out previously. In two-dimensional systems, 
the orbital nematic order will occur above the superconduct- 
ing transition, since the Ising-type quadrupole order is more 
robust than the XY-type superconducting order against ther- 
mal and quantum fluctuations. 

Up to now, we focused on the weak interaction cases, in 



which the RG method works very well till very low tem- 
peratures. Now, we study the orbital fluctuations for larger 
Coulomb interactions. In Fig. [4] (d), the T dependencies of 
the quadrupole and spin susceptibilities are shown for U = 
V > 2. Here, x q ' s is obtained by solving Fig. Q] (d) with 
using the initial values xf=a — X 0c - I n this method, al- 
though the obtained values are underestimated, we can focus 
on the T dependence originating from the correlation effects 
within Ao. In Fig. @] (d), x q (0) is enhanced by U even at 
higher temperatures, which cannot be attributed to the super- 
conducting fluctuations. This enhancement originates from 
large r quad in higher energy region A; > 0.05 in Fig.|4](c). 
Both x q (0) an d X s (Q) show similar Curie-Weiss behaviors, 
indicating that the enhancement of x q (0) originates from the 
spin fluctuations. Therefore, the magnetic AL term in Fig. @] 
(a) is the natural origin of the enhancement of x q (0) at higher 
temperatures. As the origin of the field-induced magnetic 
quantum criticality, both the van-Hove singularity 191 41211 and 
the field-suppression of quantum fluctuation 113 111 mechanisms 
had been discussed previously. 

In summary, the orbital Pomeranchuk instability driven by 
superconducting and magnetic quantum criticalities had been 
confirmed by the RG method. In the weak-interaction case 
(U < 2), where the RG method is trusted well, x q (0) is crit- 
ically enhanced by the AL-type VC due to the superconduct- 
ing fluctuations. In addition, we studied the strong-interaction 
case, and also found the development of x q (0) driven by the 
spin fluctuations, consistently with the perturbation analysis 
in Ref. 27. Both mechanisms would contribute to the nematic 
phase in Sr3Ru207, although magnetic AL term might be 
more important under the strong magnetic field. The present 
mechanisms of orbital nematic phase would be realized in var- 
ious multiorbital systems. 

The authors thank fruitful discussions with Mr. Y. Ohno and 
Prof. D.S. Hirashima. This work was supported by Grant-in- 
Aid for Scientific Research from the Ministry of Education, 
Culture, Sports, Science, and Technology, Japan. 



[1] S. A. Grigera, R. S. Perry, A. J. Schofield, M. Chiao, S. R. 

Julian, G. G. Lonzarich, S. I. Ikeda, Y. Maeno, A. J. Millis, and 

A. P. Mackenzie, Science 294, 329 (2001). 
[2] R. Perry, L. Galvin, S. Grigera, L. Capogna, A. Schofield, 

A. Mackenzie, M. Chiao, S. Julian, S. Ikeda, S. Nakatsuji, et al., 

Phys. Rev. Lett. 86, 2661 (2001). 

S. Grigera, R. Borzi, A. Mackenzie, S. Julian, R. Perry, and 
Y. Maeno, Phys. Rev. B 67, 214427 (2003). 
A. P. Mackenzie, J. A. N. Bruin, R. A. Borzi, A. W. Rost, 
and S. A. Grigera, Physica C 481, 207 (2012), and references 
therein. 

[5] K. Kitagawa, K. Ishida, R. Perry, T. Tayama, T. Sakakibara, and 

Y. Maeno, Phys. Rev. Lett. 95 (2005). 
[6] S. A. Grigera, P. Gegenwart, R. A. Borzi, F. Weickert, A. J. 

Schofield, R. S. Perry, T. Tayama, T. Sakakibara, Y. Maeno, 

A. G. Green, et al., Science 306, 1 154 (2004). 
[7] R. A. Borzi, S. A. Grigera, J. Farrell, R. S. Perry, S. J. S. Lis- 



[3] 
[4] 



5 



ter, S. L. Lee, D. A. Tennant, Y. Maeno, and A. P. Mackenzie, 

Science 315,214 (2007). 
[8] C. J. Halboth and W. Metzner, Phys. Rev. Lett. 85, 5162 (2000). 
[9] H. Yamase and A. A. Katanin, J. Phys. Soc. Jpn. 76, 073706 

(2007). 

[10] H. Yamase, Phys. Rev. B 80, 115102 (2009). 
[11] M. H. Fischer and M. Sigrist, Phys. Rev. B 81, 064435 (2010). 
[12] H. Yamase and P. Jakubczyk, Phys. Rev. B 82, 1551 19 (2010). 
[13] Y. Yoshioka and K. Miyake, J. Phys. Soc. Jpn. 81, 023707 
(2012). 

[14] C. Honerkamp and M. Salmhofer, Phys. Rev. B 64, 184516 
(2001). 

[15] C. Honerkamp, Phys. Rev. B 72, 115103 (2005). 

[16] S. Raghu, A. Paramekanti, E. Kim, R. Borzi, S. Grigera, 

A. Mackenzie, and S. A. Kivelson, Phys. Rev. B 79, 214402 

(2009). 

[17] W.-C. Lee and C. Wu, Phys. Rev. B 80, 104438 (2009). 
[18] K. W. Lo, W.-C. Lee, and P. W. Phillips. larXiv: 1207.42061 
[19] T. Takimoto, Phys. Rev. B 62, 14641 (2000). 
[20] J.-H. Chu, J. G. Analytis, K. De Greve, P. L. McMahon, Z. Is- 
lam, Y. Yamamoto, and I. R. Fisher, Science 329, 824 (2010). 



[21] M. Yoshizawa, D. Kimura, T. Chiba, S. Simayi, Y. Nakanishi, 

K. Kihou, C.-H. Lee, A. Iyo, H. Eisaki, M. Nakajima, et al., J. 

Phys. Soc. Jpn. 81, 024604 (2012), and references therein. 
[22] R. Fernandes, L. VanBebber, S. Bhattacharya, P. Chandra, 

V. Keppens, D. Mandrus, M. McGuire, B. Sales, A. Sefat, and 

J. Schmalian, Phys. Rev. Lett. 105, 157003 (2010). 
[23] S. Onari and H. Kontani, larXiv: 1203.28741 
[24] C. Halboth and W. Metzner, Phys. Rev. B 61, 7364 (2000). 
[25] T. Nishine, M. Tsuchiizu, and Y. Suzumura, J. Phys.: Conf. Ser. 

132, 012020 (2008). 
[26] H. Kontani, T. Saito, and S. Onari, Phys. Rev. B 84 (201 1). 
[27] Y. Ohno, M. Tsuchiizu, S. Onari, and H. Kontani, arXiv (2012). 
[28] R. Duprat and C. Bourbonnais, Eur. Phys. J. B 21, 219 (2001). 
[29] H. Nelisse, C. Bourbonnais, H. Touchette, Y. M. Vilk, and 

A. M. Tremblay, Eur. Phys. J. B 12, 351 (1999). 
[30] We have verified that the similar scaling behavior of p c i uad to 

Fig.|4](c) is obtained by the T-flow RG scheme fMll . 
[31] K. Sakurazawa, H. Kontani, and T. Saso, J. Phys. Soc. Jpn. 74, 

271 (2005). 



